MultiView Localization

The MultiView module is part of the CAD localization engine. It takes scans of the scene from several viewpoints, merges them into one point cloud and runs the localization on it. The merged point cloud is more complete and shows more detail, so objects in difficult scenes are found more reliably. MultiView works in both Bin Picking Studio (BPS) and Locator Studio (LS) and needs the MultiView Module software module.

There are two configurations:

  • Static MultiView – several 3D sensors are mounted in fixed positions. Each sensor takes one scan of the scene.

  • Hand-eye MultiView – one 3D sensor is mounted on the robotic arm. The robot moves the sensor and it takes all the scans.

Static MultiView

Static MultiView vision system with three PhoXi 3D Scanners

Example of a Static MultiView vision system with three PhoXi 3D Scanners.

Configuration

To create a Static MultiView vision system:

  1. Set Calibration space and sensor position to Robot space, external fixed sensor position.

  2. Add more than one 3D sensor to the vision system.

The first sensor you add is the primary sensor. The other sensors are secondary sensors.

Calibration

A Static MultiView vision system needs two calibrations:

  1. Robot-camera calibration of the primary sensor to the robot.

  2. Multi-sensor calibration of all secondary sensors to the primary sensor. To start it, click Calibrate together below the sensors.

Deployment

During deployment, the robot program starts a scan with the Scan action request (Bin Picking Studio / Locator Studio). Then it waits for the result with Wait for scan completion.

Hand-eye MultiView

Hand-eye MultiView vision system with a MotionCam-3D

Example of a Hand-eye MultiView vision system with a MotionCam-3D.

Configuration

To create a Hand-eye MultiView vision system:

  1. Set Calibration space and sensor position to Robot space, sensor mounted on the robotic flange.

  2. Turn on the Hand-eye MultiView toggle.

  3. Select the mode:

    • Static – the robot must stand still for every scan.

    • Dynamic – the robot must stand still for the first scan. The next scans can be taken while the robot is moving, which needs a MotionCam-3D in Camera mode.

  4. Select a 3D sensor that suits the mode.

Calibration

A Hand-eye MultiView vision system uses the standard hand-eye robot-camera calibration. See Calibration.

Deployment

During deployment, the robot program adds the scans one at a time:

  1. For each scan, send the Trigger scan action request (Bin Picking Studio / Locator Studio), followed by Wait for scan completion.

  2. When all scans are taken, send the standard Scan action request (Bin Picking Studio / Locator Studio). The Studio merges all added scans into one virtual point cloud and runs the localization. In a bin picking solution, it also runs the path planning.

Animation of a robot moving a 3D sensor to three poses above a bin; each scan adds the object surfaces it sees, then the scans are merged

Hand-eye MultiView in static mode: the robot stops at three poses and the robot program sends a Trigger scan at each. Each color marks the surfaces one scan sees. The Scan request merges them into one point cloud.

What each Trigger scan request must contain depends on the mode:

  • Static mode – every Trigger scan request must contain the robot TCP. Make sure the robot stands still and the TCP values are settled when you send the request.

  • Dynamic mode – only the first Trigger scan request must contain the robot TCP. Make sure the robot stands still and the TCP values are settled when you send it. The next requests don’t need the TCP. If they contain it, it is ignored.